This application claims the benefit of a Japanese Patent Application No.2002-193707 filed Jul. 2, 2002, in the Japanese Patent Office, the disclosure of which is hereby incorporated by reference.
1. Field of the Invention
The present invention generally relates to image distortion correcting methods and apparatuses and storage media for correcting distortion within an input image related to a document or book imaged (or picked up) by a digital camera, and more particularly to an image distortion correcting method for correcting the distortion within the input image by estimating an imaging position of the camera, an image distortion correcting apparatus which corrects the distortion within the input image by such an image distortion correcting method, and a computer-readable storage medium which stores a computer program for causing a computer to make such an image distortion correction.
2. Description of the Related Art
Recently, digital cameras have become popular due to improved resolution and reduced cost. Hence, the digital camera is regarded as one suitable tool for inputting paper document information to a computer.
When the digital camera is used to input the paper document information to the computer, the user has more flexibility compared to the case where a conventional contact type image input apparatus such as an image scanner is used, because the user can input the necessary information on the paper to the computer while inspecting the paper or writing on the paper.
On the other hand, unlike a scanned image obtained by the scanner, the image obtained by the digital camera includes a three-dimensional distortion of the image. FIG. 1 is a diagram showing an image 200 which is obtained by imaging a flat paper surface from an oblique angle by the digital camera. In this case, since the paper surface is imaged from the oblique angle from above, a perspective transformation distortion caused by perspective transformation is generated in the image 200.
FIG. 2 is a diagram showing an image 202 which is obtained by imaging a relatively thick book, such as a dictionary, encyclopedia and handbook, which is opened, from an oblique angle by the digital camera. In this case, since the book is imaged from the oblique angle from above, a perspective transformation distortion caused by perspective transformation is generated in the image 202. In addition, a distortion caused by a curved paper surface of the opened book is also generated in the image 202. A composite distortion made of this latter distortion and the perspective transformation distortion will hereinafter be referred to as a “book distortion” in this specification.
In order to input the paper document information to the computer using the digital camera, it is necessary to correct image distortions such as the perspective transformation distortion shown in FIG. 1 and the book distortion shown in FIG. 2, so as to generate a paper surface image 204 shown in FIG. 3 having no distortion. FIG. 3 is a diagram showing the paper surface image 204 which is obtained by imaging a paper document from an oblique angle by the digital camera but including no distortion. Accordingly, there are demands to develop a distortion correcting technique which can generate the paper surface image 204 having no distortion.
For example, a paper document may be bound or stapled at one corner. With respect to such a paper document bound or stapled at one position, an image reading apparatus has been proposed in a Japanese Laid-Open Patent Application No. 10-150532 to correct the image distortion. According to this proposed image reading apparatus, a three-dimensional shape of a contour line is used to measure a three-dimensional distortion of the paper surface. The three-dimensional shape is obtained from an image which is imaged by a scan of a line sensor which uses a CCD array.
In addition, a method of correcting a three-dimensional distortion with respect to a form sheet based on an image which is imaged from above has been proposed in the Japanese Laid-Open Patent Application No. 10-150532.
However, according to the image reading apparatus proposed in the Japanese Laid-Open Patent Application No. 10-150532, there was a problem in that an expensive apparatus is required exclusively for estimating the three-dimensional shape of the contour of the paper surface within the image. In a state where the document or book is placed on a document base, this exclusive apparatus generates a two-dimensional image by imaging line images of the paper surface at a plurality of positions by mechanically moving a line sensor which is arranged above the document base in a sub scan direction while electrically scanning in a main scan direction which is perpendicular to the sub scan direction.
Furthermore, the distortion correcting method proposed in the Japanese Laid-Open Patent Application No. 10-150532 has an advantage in that the distortion can be corrected based on a single image. However, an area CCD imaging element is used as an overhead reader (OHR) and fixed to a stand, so as to image the form sheet positioned below. Hence, it is a precondition that a center of the camera is determined in advance. This is also the case for the image reading apparatus proposed in the Japanese Laid-Open Patent Application No. 10-150532.
Therefore, there was a problem in that the image reading apparatus proposed in the Japanese Laid-Open Patent Application No. 10-150532 and the distortion correcting method proposed in the Japanese Laid-Open Patent Application No. 10-150532 cannot cope with a situation where the imaging position is unknown, such as a case where the user obtains the image of the paper document by the digital camera or the like from an arbitrary position.